360° mulsemedia experience over next generation wireless networks - a reinforcement learning approach

The next generation of wireless networks targets aspiring key performance indicators, like very low latency, higher data rates and more capacity, paving the way for new generations of video streaming technologies, such as 360° or omnidirectional videos. One possible application that could revolutionize the streaming technology is the 360° MULtiple SEnsorial MEDIA (MULSEMEDIA) which enriches the 360° video content with other media objects like olfactory, haptic or even thermoceptic ones. However, the adoption of the 360° Mulsemedia applications might be hindered by the strict Quality of Service (QoS) requirements, like very large bandwidth and low latency for fast responsiveness to the users, inputs that could impact their Quality of Experience (QoE). To this extent, this paper introduces the new concept of 360° Mulsemedia as well as it proposes the use of Reinforcement Learning to enable QoS provisioning over the next generation wireless networks that influences the QoE of the end-users

Seamless multimedia delivery within a heterogeneous wireless networks environment: are we there yet?

The increasing popularity of live video streaming from mobile devices such as Facebook Live, Instagram Stories, Snapchat, etc. pressurises the network operators to increase the capacity of their networks. However, a simple increase in system capacity will not be enough without considering the provisioning of Quality of Experience (QoE) as the basis for network control, customer loyalty and retention rate and thus increase in network operators revenue. As QoE is gaining strong momentum especially with increasing users’ quality expectations, the focus is now on proposing innovative solutions to enable QoE when delivering video content over heterogeneous wireless networks. In this context, this paper presents an overview of multimedia delivery solutions, identifies the problems and provides a comprehensive classification of related state-of-the-art approaches following three key directions: adaptation, energy efficiency and multipath content delivery. Discussions, challenges and open issues on the seamless multimedia provisioning faced by the current and next generation of wireless networks are also provided

An innovative machine learning-based scheduling solution for improving live UHD video streaming quality in highly dynamic network environments

The latest advances in terms of network technologies open up new opportunities for high-end applications, including using the next generation video streaming technologies. As mobile devices become more affordable and powerful, an increasing range of rich media applications could offer a highly realistic and immersive experience to mobile users. However, this comes at the cost of very stringent Quality of Service (QoS) requirements, putting significant pressure on the underlying networks. In order to accommodate these new rich media applications and overcome their associated challenges, this paper proposes an innovative Machine Learning-based scheduling solution which supports increased quality for live omnidirectional (360◦) video streaming. The proposed solution is deployed in a highly dy-namic Unmanned Aerial Vehicle (UAV)-based environment to support immersive live omnidirectional video streaming to mobile users. The effectiveness of the proposed method is demonstrated through simulations and compared against three state-of-the-art scheduling solutions, such as: Static Prioritization (SP), Required Activity Detection Scheduler (RADS) and Frame Level Scheduler (FLS). The results show that the proposed solution outperforms the other schemes involved in terms of PSNR, throughput and packet loss rate

5MART: A 5G SMART scheduling framework for optimizing QoS through reinforcement learning

The massive growth in mobile data traffic and the heterogeneity and stringency of Quality of Service (QoS) requirements of various applications have put significant pressure on the underlying network infrastructure and represent an important challenge even for the very anticipated 5G networks. In this context, the solution is to employ smart Radio Resource Management (RRM) in general and innovative packet scheduling in particular in order to offer high flexibility and cope with both current and upcoming QoS challenges. Given the increasing demand for bandwidth-hungry applications, conventional scheduling strategies face significant problems in meeting the heterogeneous QoS requirements of various application classes under dynamic network conditions. This paper proposes 5MART, a 5G smart scheduling framework that manages the QoS provisioning for heterogeneous traffic. Reinforcement learning and neural networks are jointly used to find the most suitable scheduling decisions based on current networking conditions. Simulation results show that the proposed 5MART framework can achieve up to 50% improvement in terms of time fraction (in sub-frames) when the heterogeneous QoS constraints are met with respect to other state-of-the-art scheduling solutions

LearnSDN: optimizing routing over multimedia-based 5G-SDN using machine learning

With the advent of 5G networks and beyond, there is an increasing demand to leverage Machine Learning (ML) capabilities and develop new and innovative solutions that could achieve efficient use of network resources and improve users' Quality of Experience (QoE). One of the key enabling technologies for 5G networks is Software Defined Networking (SDN) as it enables fine-grained monitoring and control of the network. Given the variety of dynamic networking conditions within 5G-SDN environments and the diversity of routing algorithms, an intelligent control of these strategies should exist to maximize the Quality of Service (QoS) provisioning of multimedia traffic with more stringent requirements without penalizing the performance of the background traffic. This paper proposes LearnSDN, an innovative ML-based solution that enables QoS provisioning over multimedia-based 5G-SDN environments. LearnSDN uses ML to learn the most convenient routing algorithm to be employed on the background traffic based on the dynamic network conditions in order to cater for the QoS requirements of the multimedia traffic. The performance of the proposed LearnSDN solution is evaluated under a realistic emulation-based SDN environment. The results indicate that LearnSDN outperforms other state-of-the-art solutions in terms of QoS provisioning, PSNR and MOS

Performance evaluation of routing strategies over multimedia-based SDNs under realistic environments

Most of the existing performance evaluation studies of various routing algorithms are done under limited experimental setups leading to an incomplete picture of the routing algorithm performance under dynamic network conditions. This paper presents a study that compares state-of-the-art routing algorithms over realistic multimedia-based Software Defined Networks (SDNs) with dynamic network conditions and various topology. Routing algorithms remain a key element of the networking landscape as they determine the path the data packets follow. The next-generation networking paradigm offers wide advantages over traditional networks through simplifying the management layer, especially with the adoption of SDN. However, Quality of Service (QoS) provisioning still remains a challenge that needs to be investigated especially for multimedia-based SDNs. This study investigates the impact of state-of-the-art centralized routing algorithms (e.g. MHA, WSP, SWP, MIRA) on multimedia QoS traffic u nder a realistic environment in terms of PSNR, Throughput, Packet Loss, Delay and QoS rejection

REDO: a reinforcement learning-based dynamic routing algorithm selection method for SDN

The current increase in the Internet traffic along with the global crisis have accelerated the roll-out of the next generation 5G network and key enabling technologies. In this context, addressing the end-to-end Quality of Service (QoS) provisioning in order to guarantee a sustainable service delivery to the end-users became of paramount importance. Some of the enabling technologies that could play a key role in this regard are Software Defined Network (SDN) and Machine Learning (ML). This paper proposes REDO, a Reinforcement lEarning-based Dynamic rOuting algorithm selection method that decides on the conventional routing algorithm to be applied on the traffic flows within a SDN environment. REDO will dynamically select the most appropriate routing algorithm from a set of centralized routing algorithms (MHA, WSP, SWP, MIRA) that maximizes the reward function from the network. The proposed REDO solution is implemented and evaluated using an experimental setup based on Mininet, Floodlight controller and Open vSwitch switches. The results show that REDO outperforms other state-of-the-art solutions

An innovative reinforcement learning-based framework for quality of service provisioning over multimedia-based SDN environments

Within the current global context, the coronavirus pandemic has led to an unprecedented surge in the Internet traffic, with most of the traffic represented by video. The improved wired and guided network infrastructure along with the emerging 5G networks enables the provisioning of increased bandwidth support while the virtualization introduced by the integration of Software Defined Networks (SDN) enables traffic management and remote orchestration of networking devices. However, the popularity and variety of multimediarich applications along with the increased number of users has led to an ever increasing pressure that these multimedia-rich content applications are placing on the underlying networks. Consequently, a simple increase in the system capacity will not be enough and an intelligent traffic management solution is required to enable the Quality of Service (QoS) provisioning. In this context, this paper proposes a Reinforcement Learning (RL)-based framework within a multimedia-based SDN environment, that decides on the most suitable routing algorithm to be applied on the QoS-based traffic flows to improve QoS provisioning. The proposed RL-based solution was implemented and evaluated using an experimental setup under a realistic SDN environment and compared against other state-of-the-art solutions from the literature in terms of throughput, packet loss, latency, peak signal-to-noise ratio (PSNR) and mean opinion score (MOS). The proposed RL-based framework finds the best trade-off between QoS vs. Quality of User Experience (QoE) when compared to other state-of-the-art approaches

A machine learning resource allocation solution to improve video quality in remote education

The current global pandemic crisis has unquestionably disrupted the higher education sector, forcing educational institutions to rapidly embrace technology-enhanced learning. However, the COVID-19 containment measures that forced people to work or stay at home, have determined a significant increase in the Internet traffic that puts tremendous pressure on the underlying network infrastructure. This affects negatively content delivery and consequently user perceived quality, especially for video-based services. Focusing on this problem, this paper proposes a machine learning-based resource allocation solution that improves the quality of video services for increased number of viewers. The solution is deployed and tested in an educational context, demonstrating its benefit in terms of major quality of service parameters for various video content, in comparison with existing state of the art. Moreover, a discussion on how the technology is helping to mitigate the effects of massively increasing internet traffic on the video quality in an educational context is also presented

A comparison of reinforcement learning algorithms in fairness-oriented OFDMA schedulers

Due to large-scale control problems in 5G access networks, the complexity of radioresource management is expected to increase significantly. Reinforcement learning is seen as apromising solution that can enable intelligent decision-making and reduce the complexity of differentoptimization problems for radio resource management. The packet scheduler is an importantentity of radio resource management that allocates users’ data packets in the frequency domainaccording to the implemented scheduling rule. In this context, by making use of reinforcementlearning, we could actually determine, in each state, the most suitable scheduling rule to be employedthat could improve the quality of service provisioning. In this paper, we propose a reinforcementlearning-based framework to solve scheduling problems with the main focus on meeting the userfairness requirements. This framework makes use of feed forward neural networks to map momentarystates to proper parameterization decisions for the proportional fair scheduler. The simulation resultsshow that our reinforcement learning framework outperforms the conventional adaptive schedulersoriented on fairness objective. Discussions are also raised to determine the best reinforcement learningalgorithm to be implemented in the proposed framework based on various scheduler settings